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ABSTRACT 



This paper 1 describes the construction cf a new 
examination for Federal Coal Mine Inspectors, using the Job Element 
(J— Scale) method. Headquarters administrators and field inspectors 
rated the importance of various job elements. The values given to 32 
elements, a comparison of test and actual proficiency, ratings, and 
examples of the rating parameters of several elements are given. A 
summary of the workings oi the J— Scale Formula is provided. See also 
TM 001 163, 165-166 for further information on the Job Element 

(J— Scale) method. {DLG) 




HANDOUT 4 
TOPIC V 



o|o; 

;jj U- O 1 



£§S 

> P § 

uj ^ 
U N ^ 

LU p- ■> 

DC ^ u_ 
w oO 

< o£ 

iccE 

i-OO 

(_> _ Q- 

1 o i_ 

i x 5 t= 

LU CO 

1 cn o 
1 Q uj z 

!is§ 



5 3 

e q 
< ^ 



tn O 

LU 

O ft >. 

Z u- — 
u- — ^ 
jog 

z «g 

o o ^ 

°Ei 

O O t 
£ i- ft 

2£S 

"So 

l/> DC rj 

Q- J— 

o £ < 



APPLICATION OF JOB ELEMENT (J-SCALE) METHOD TO JOB 
ANALYSIS AND SELECTION OF INSPECTORS 

Ernest S. Primoff, Personnel Measurement Research and 
Development Center, Standards Division 
U. S. Civil Service Commission 

Delivered at Meeting of Division 18, A. P. A., Friday morning, 
August 29, 1958, 11:30 A.M. 
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A Federal Coal Mine Inspector has the responsibility for closing an un- 
safe mine to save lives, in spite of the economic impact of closing a mine 
on owners and community. His most important contribution is in cooperative 
work with mine officials and miners to improve safety conditions in mines. 

In previous examinations, applicants were rated in terms of length of 
experience in positions such as mine superintendent, mine foreman or mine 
engineer. This type of examination made it impossible to hire a person of 
considerable ability until his ability had been recognized by a mine com- 
pany. Also, a miner who preferred to keep a nonsupervisory job, which in- 
cidentally often pays as much money to a good producer as a supervisor 
makes, could not aspire to a Federal Coal Mine Inspe’Ctsor ’ s position. 



A new examination was set up, to permit spplicarsxs to demonstrate com- 
petence in necessary job elements, regardless of how they acquired compe- 
tence — by combinations of home study, school study, cn-the-job training, etc. 

Ihis examination was based on the Job-Element a proach , which makes use 
of a validated procedure for analyzing a job in terns of the qualifications 
that will select the best possible employees. 
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The first step was to find out what the important qualification ele- 
ments for the position are. In a round-table conference with nine Coal Mine 
Inspector Supervisors of the Bureau of Mines headquarters stall*, a comprehen- 
sive list of knowledges, skills and personal characteristics required by the 
position were described, such as "ability to learn to n&ke acceptable inspec- 
tion reports," "fundamental knowledge of mine gases and ventilation," "stamina 
for work in mining," "ability to obtain cooperation and to cooperate," and 
"resoluteness to carry out judgment." 



The next step was to determine the relative value of each qualification 
element for an examination. In early studies, this was a lengthy process of 
extended conferences wi*th experts, a considerable number of unresolved argu- 
ments , and often indefinite conclusions. By validation studies (comparing 
opinions of experts with later successful or unsuccessful use of elements in 
examining), the J-scale method was evolved, in which the known facts about a 
r- position are rapidly indicated on a check sheet by experts, on the basis of 

CsP their experience and judgnent, and conclusions are made by applying a formula 

based on research. 
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Ihe nine headquarters administrators and also six 
filled out the check list shown in Chart 1. 



inspectors in “the field 
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CHART Is JOB ANALYSIS BLANK 



Used to Determine Total 
J-scalo Value of Each Element 



rob 

Element 

■Jo. 


To be barely 
acceptable worker 

+ Must have 
V Good to have 

0 Little value 


To pick out 
superior worker 

+ Essential 

\J Valuable 

0 Does not 

differentiate 


Trouble likely 
if ignored 

+ Much trouble 

\/ Some trouble 

0 Safe to ignore 


Practical to 
expect 

+ For all 
eligibles 
\J For some 
eligibles 
0 Almost no one 











































J-scale formula, applied to Coal Mine Inspector Examination 

+ = 2 
V * 1 
0=0 

Total Value for a rater - Barely acceptable + Trouble likely + (Superior x 

Practical) 

Information on the development ana use of the J-scale formula can be secured 
from the author. See Appendix for a quick view of how the formula works. 

For each element, each rater makes four simple decisions. If the element is 
extremely important for even barely acceptable work, he marks a + in the "Barely 
acceptable" column. If the element will distinguish superior workers, the rater 
puts a + in the " Superior " column. If trouble on the job is likely if we ignore 
the element in an examination, he puts a + in the "Trouble likely" column. If the 
qualification is practical to expect in aLl people hired, he puts a + in the 
"Practical" column. Similarly, the rater may mark a 0 or \/ if he feels that 
the element is of less importance. 

Each rater makes a fairly simple determination — plus, check or zero in each 
column. Ihe J-scale formula is applied later, by a clerk, to secure a total value. 
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Chart 2, taken from the Link-Belt News, shows the advantage of rapid decision- 
making techniques like this one. 



CHART 2; HOW DECISIONS ARE REACHED 
FROM: LINK-BELT NEWS APRIL-MAY 1958 
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others 
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JUDGMENT 
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Wishful Thinking 
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Day Dreaming 
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CONCLUSION 
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Chart- 3 shows the elements that have highest total values Tor the entry grade 
and were selected for the examination, the total value of each such element, and 
how the element is measured: 

CHART 3: Total J-scale Values for Elements Selected for Examina- 

tion for Entry Grade (GS-9), and Means of Measurement 



Element 


J-Scale 
Total Value 


How Measured in 
Examination 


1. Physical fitness for work in 
mining 


36 


Medical examination 


2. Stamina for work in mining 


36 


Performance test 


3. Ability to interpret written 
instructions and regulations 


36 


Experience evaluation 


U. Learning ability for regula- 
tions and procedures 


36 


Experience evaluation 5 
Written tesi. 


9. Ability to accept supervision 


36 


Experience evaluation 


6 . Ability to interpret and apply 
instructions given orally 


32 


Experience evaluation; 
Interview 


7* Ability to work as a member of 
a team 


32 


Experience evaluation 


8 . Ability to learn to make ac- 
ceptable inspect! on reports 


32 


j 

Experience evaluation; 

Written test j 


9* Ability to make acceptable in- 
spection reports 


1 / 


f 

t 

j 


10. Ability to observe, recognize 
and interpret what is seen 


32 


Experience evaluation; 
Written test 


11. Ability to drive an automobile 


36 


Performance test 


12 . Satisfactory personal conduct 
and integrity 


32 


Experience evaluation 2 / 



1 / Elem^it 9 had a low value for GS-9, although it had a high value for higher 
grades. The element was not counted for the examination for GS-9 positions. 



2 / Experience is interpreted broadly to include relationships with people and 
society, outside as well as inside t 1 ob areas. 
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Flement 

13 » Emotional stability 

lh. Fundamental knowledge of 
methods of mining 

1?. Fundamental knowledge of mine 
gases and ventilation 

16. Fundamental knowledge of roof- 
contro.i methods 

17. Fundamental knowledge of proper- 
ties and use of explosives 

18. Knowledge of arithmetic as ap- 
plied to mine safety 

19* Fundamental knowledge of mine 
tra nsp or ta t i on 

20. Knowledge of machinery, funda- 
mental mechanics, and electric- 
ity 

21. Fundamental knowledge of fire 
control 

22. Fundamental knowledge of first 
aid and mine rescue 

23. Judgment to determine correct 
from incorrect courses of action 

2lu Ability to obtain cooperation 
and to cooperate 

25. Willingness to refer complex 
situations to higher authority 

26. Resoluteness to carry out 
judgnent 

27 • Tact and diplomacy 

28. Ability to interpret mine 
drawings and maps 



J-Scale f 
32 

32 



32 

32 

28 

28 

28 

20 

16 

12 

28 

28 

28 

28 

2k 

2h 






How Measured _ 
Examination 



in 



Experience evaluation; 
Interview 

Experience evaluation; 
Written test 

Experience evaluation; 
Written test 

Experience evaluation; 
Written test 

Experience evaluation; 
Written test 

Exp erie nee evalu ation 5 
Written test 

Experience evaluation; 
Written test 

Experience evaluation; 
Written test 



Experience evaluation; 
Written test 

Experience evaluation; 
Written test 

Experience evaluation; 
Written test 

Experience evaluation 



Experience evaluation 



Experience evaluation 



Experience evaluation; 
Interview 

Experience evaluation; 
Written test 
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J-Scale How Measured in 

Total Value Examination 



'9. Ability to use inspection 
instruments 


2b 


Experience evaluation; 
Writ 'ten test 


!0. Neatness in dress and 


16 


Intervi ew 


appearance 






tl. Ability to promote public rela- 
tions 


20 


Experience evaluation; 
Interview 


52. Ability to express himself 
orally 


20 


Experience evaluation; 
Interview 



A written test was prepared with the assistance of Mr. Charlesworth, Mr, 
Schrecengost, and others of the Bureau of Mines, for appropriate elements, as indi- 
cated in Chart 3. Harvey Pearce of the Bureau of Mines validated test units with 
GS-12 inspectors for some individual elements and for the whole job. For example, 
for the elements: "Ability to learn to make acceptable inspection reports, " a 

multiple-choice test of ability to organize a report paragraph was prepared. Of 
fifteen inspectors, three were rated above-average in report-writing ability; all 
three rea *d U3 in the test. Only one other inspector scored as higi; he was rated 
average in report-writing ability. Three inspectors were rated below-average in 
report-writing ability, the highest score any of these achieved was UO, three of eight 
inspectors rated average, and all rated above-average scored higher. Following are 
results for a test of another element, "Ability to interpret mine drawings and maps. 
All five inspectors rated above-average in understanding scored 20 or over, h of the b 
inspectors rated average scored as high, all U inspectors rated below-average scored 
lower. Chart U shows results for the test as a whole. 

CHART U: Scores for Complete Coal Mine Inspector 

Test, administered to inspectors of senior 
grade who were evaluated in job perform- 
ance. 



Test Score 


Above-Average 
in Job Prof ic ienc;^ 


Average in 
Job Proficiency 


Below- Average 
in Job Proficiency 


1U6- 


1 




Ikl-lhZ 


1 


1 




136-lUo 


111 


11 




131-135 




11 




126-130 




1 


111 


121-125 






1 
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As indicated in Chart U, the employed inspectors who were considered by their 
ipervisor to be abore-average in job proficiency all got scores over 136. No em- 
loyed inspector scored below 121 . 

For the competitors, a minimum score in the test of 120 is required so that 
the competitor will be about as high in the test as the lowest scoring inspector. 

It is realized, of course, that the actual inspectors in the try-out have the 
advantage of experience. Special forms have been devised for competitors to show 
their experience, and for supervisors and others acquainted with the competitors 
to report what they know of the competitors. A group of inspector supervisors have 
been trained as interviewers. 

A competitor's final rating is based on a combination of written test scores 
and experience evaluation. In evaluating experience, the raters have before them 
the complete record of each competitor — the experience he claimed, his test scores, 
the results of interviews, and reports from supervisors and others who know the 
competitor. 

Each of two raters makes a simple determination for each element, as to whether 
a competitor is worth full credit, 3/U credit, 1/2 credit, l/k credit, or zero 
credit in each element. For this purpose, rating guides are used. The conduct of 
the examination, including preparation of rating guides, etc., was the responsibility 
of Mr. Nathan Shinderman, of the Commission’s Bureau of Departmental Operations and 
Mr. Harvey Pearce, Executive Secretary of the Civil Service Board of Examiners for 
the Bureau of Mines. Under their direction, Mr. Arthur Charlesworth and Mr. Harry 
Schrecengost as Board members, developed the drafts of the guides and related mate- 
rial. Mrs. Henrietta Moore of the Commission assisted in the training of raters and 
interviev/ers for the examination. 

Chart 5 shows several examples of rating guides. The rating guide for element 
8 in the chart, that for "Ability to learn to make acceptable inspection reports," 
illustrates how test scores are balanced with proven experience for elements re- 
lated to learning ability. As we have already mentioned, inspectors who are known 
to be excellent writers of reports do get high scores in the written test of this 
ability. However, it might happen in usual cases that a person with excellent 
proven experience might fail the report-writing test. Therefore, we give full 
credit both to a person who can demonstrate his report-writing ability in a test, 
even though he has not had an opportunity to aply his ability on a job, and to a 
person who has demonstrated his ability on a job. 

Chart 5. Examples of Rating Guides (with Directions to Examiners for 
their use ) 

A. DIRECTIONS TO EXAMINERS FOR USE OF RATING GUIDES 

Guidelines for rating the elements are given in terms of excellent performance. 
Evidence of lower quality performance may reveal lack of abilities, knowledges, 
and skills normally required by the kind of experience presented by a competitor. 
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'Ihe competitor should be given credit merited by his experience and 
training as they are demonstrated in the examination and by the qualities of 
his vouchers and personal interview. This may result in the evaluation of 
an element one or more values below that provided for an "excellent” evaluation. 



Carefully read each of the qualifying statements listed under the partic- 
ular element under consideration . Select that part of the element which most 
nearly describes the performance of the individual being analyzed and evaluated. 
Then, enter the corresponding value of that particular element on "the Coal Mine 
Inspector score sheet. 

B. EXAMPLES OF RATING GUIDES 

1. Ability to Work as a Member of a Team 

Consider on-the-job associations, activities in various organiza- 
tions such as Civic, Safety, Church, and Labor groups, etc. 

a. Full credit: Holds responsible elective or appointive office. 

Serves on committees involving group activities, such as Civic, 
Safety, Church, and Labor groups, etc. 

b. l/U credit: Serves in routine positions; records of employment and 

reputation in the community do not contain any derogatory informa- 
tion. 

2. Ability to Learn to Make Acceptable Inspection Reports 

The element is self-explanatory 

a. Full credit: Vhen applicant indicates actual preparation and writ- 
ing of technical reports has comprised a major portion of his jcto. 

OR candidate gets U3 or higher in Test No. 5>05. 

b. 3/U credit: When applicant's duties required actual preparation 

and writing of technical reports which were incidental to job or 
was responsible for reviewing technical reports. OR candidate 
gets 38 to 1*2 in Test No. 50J?. 

c. 1/2 credit: When applicant's duties required preparation and writ- 

ing of reports containing little detail. OR candidate gets 33 to 
37 in Test No. $05. 

d. l/U credit: When experience of applicant shows some familiarity 

with technical reports. OR candidate gets 28 to 32 in Test No. £05. 
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3» Basic Knowledge of Roof-Control Methods 

a. Full credit : Wien applicant has ability equal to that of 

supervisor who has successfully completed courses including 
mine roof -control methods and has had wide experience in 
supporting various kinds of mine roof with roof bolts and 
conventional timbering during advance mining and pillar 
recovery operations, and scores at least 10 in items 1 
thrcaigh 10 in Test No. 502. 

b. 3/h credit : When applicant has ability equal to that of 

supervisor who has successfully completed courses including 
mine roof-control methods and has had wide experience in 
supporting various kinds of mine roof with conventional 
timbering during advance mining and pillar recovery opera- 
tions, and scores at least 9 in items ' through 10 in Test 
No. 502. 

c. l/2 credit : When experience has been gained as timberman 

or roof— bolter, and scores at least 8 in items 1 through 10 
in Test No. 502. 

d. l/h credit ; Wien applicant has had limited experience in 
roof-control methods, and scores at least 7 in items 1 through 
10 in Test Nc* 502. 

U. Fundamental Knowledge of Properties and Use of Explosives 

a* Full credit s Wien applicant has successfully completed courses 
including mine explosives and has had wide experience, in the 
transportation, storage, handling, and use of explosives under- 
ground. 

b. 3/U credit ; Wien applicant has successfully completed courses 
including mine explosives and has had experience in the use of 
explosives. 

c. l/2 credit: When experience has been gained as a shot firer. 

d. l/h credit: 1-hen applicant has had limited experience in the 

use of explosives. 

Mr. Marling J. Ankeny, Director of the Bureau of Mines, and Mr. James 
Westfield, Assistant Director for Health and Safety, are very highly pleased 
with eligibles who have passed the examination. As the new inspectors are 
evaluated in actual work on the job, further validity checks will be made. 
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APPENDIX 

A Quick View of How the J -Sc ale Formula Works 

A plus counts 2 j a check counts 1* A.n element that is rated plus for pick- 
ing superior workers is thereby given a credit Of 2. This consideration must, 
however , be modified by the consideration as to whether it is practical to expect. 
We don’t want to load our rating schedule with elements that will affect very 
few competitors. 

One example is trigonometry for trades positions. Supervisors often mention 
trigonometry as desirable for gage checkers and machinists, and one agency re- 
quested that we require trigonometry for apprentice applicants. In the latter 
case, a validity study showed that the rare applicants for apprentice Jobs who 
know trigonometry do reach superior performance on the job. However, there are 
usually no applicants for a particular apprentice examination who know trigonom- 
etry. If examiners had to evaluate every applicant for knowledge of trigonometry, 
they would spend a let of unnecessary time, for example, trying to evaluate work 
experience with Jo-Blocks as 0 or | credit in trigonometry. Any advantage that 
would accrue from examining for trigonometry would be gained equally by consid- 
eration given in a rating schedule that knowledge of trigonometry is worth extra 
credit in an element like advanced lay-out. 

Therefore, the rating for Practical on the four-column blank is used as a 
governor for the rating for Superior Workers. If the element is practical to 
expect for all eligibles, then it is rated plus, worth 2 # This 2 is multiplied 
by the value for Discriminating Superior Workers. If an element will pick 
superior workers and is practical to expect, it gets a product of U. If it is 
not practical to expect, it gets a product of zero. If it will pick superior 
workers and is practical to expect for only some hires, then it has 2 for 
Superior Workers and 1 for Practical to Expect, giving a product of 2. If it is 
practical to expect, but does not differentiate superior workers at all, it de- 
serves no credit and gets a product of 2 X 0 or zero. 

Vhen rating experience, examiners need to give a minimal credit for quali- 
fications that make for barely acceptable work, so as to set a floor for pass- 
ing with a minimum rating of 70. Therefore, to the above product there is added 
the value for the rating of relevance to the Barely Acceptable worker. If 
an element is likely to give trouble if igiored in the entrance requirement, it 
deserves more consideration than if it is of minor consequence. Therefore, the 
value for Trouble Likely if Ignored is also added. 

The sum of The values for an element computed for each rater makes up a to- 
tal value on a J-scale for the job ("J" for "Job"). The maximum value that can 
be given by one rater to an element is 8. With 10 raters, the maximum possible 
value is 80; thus with 10 raters, elements are evaluated on a J-scale from 0 
to 80. 

In order to avoid large numbers in actual examining, the total values are 
multiplied by an arbitrary decimal cons tent, and reduced to the nearest multiple 
of U, since examiners will rate competitors in terms of full, 3/U, 1/2, l/U, or 
0 credits. The amount of credit given by an examiner to a competitor in an 
element is in proportion to the J-scale value of the element, so that each element 
counts proportionately to its importance. 
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